OpenTS-Bench Get Started

OpenTS-Bench empowers you with three key capabilities:

  • Benchmark Your Algorithm: Evaluate your own, newly developed algorihtm/model against standardized datasets.
  • Test Your Dataset: Validate how existing algorithms perform on your custom dataset.
  • Track Algorithm Rankings: Monitor performance comparisons in different settings on our interactive leaderboard.

Table of Contents

  1. Quickstart
  2. Benchmark Your Algorithm
  3. Test Your Dataset
  4. Track Algorithm Rankings
  5. FAQ
  6. Citing us
  7. Contact

Quickstart

Installation

From PyPI

Given a python environment (note: this project is fully tested under python 3.8), install the dependencies with the following command:

1
pip install -r requirements.txt

From Docker

We also provide a Dockerfile for you. For this setup to work you need to have a Docker service installed. You can get it at Docker website.

1
docker build . -t tfb:latest
1
docker run -it -v $(pwd)/:/app/ tfb:latest bash

Data preparation

Place the downloaded data under the folder ./dataset.

Train and evaluate model

We provide the experiment scripts for all benchmarks under the folder ./scripts/multivariate_forecast, and ./scripts/univariate_forecast. For example you can reproduce a experiment result as the following:

1
sh ./scripts/multivariate_forecast/ILI_script/DLinear.sh

Benchmark Your Algorithm

We provide tutorial about how to develop your own method, you can click here.

Test Your Dataset

We provide tutorial about how to evaluate on your own time series, you can click here.

FAQ

How to use Pycharm to run code？

When running under pycharm，please escape the double quotes, remove the spaces, and remove the single quotes at the beginning and end.

Such as: ‘{“d_ff”: 512, “d_model”: 256}’ —> {\“d_ff\“:512,\“d_model\“:256}

1
--config-path "rolling_forecast_config.json" --data-name-list "ILI.csv" --strategy-args {\"horizon\":24} --model-name "time_series_library.DLinear" --model-hyper-params {\"batch_size\":16,\"d_ff\":512,\"d_model\":256,\"lr\":0.01,\"horizon\":24,\"seq_len\":104} --adapter "transformer_adapter"  --gpus 0  --num-workers 1  --timeout 60000  --save-path "ILI/DLinear"

Citing us

If you find our works helpful, please cite our paper (Bibtex below).

PVLDB 2024
Best Research Paper Nomination

PVLDB 2025

SIGKDD 2025

1
2
3
4
5
6
7
@inproceedings{qiu2024tfb,
    title   = {TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods},
    author  = {Xiangfei Qiu and Jilin Hu and Lekui Zhou and Xingjian Wu and Junyang Du and Buang Zhang and Chenjuan Guo and Aoying Zhou and Christian S. Jensen and Zhenli Sheng and Bin Yang},
    booktitle = {Proc. {VLDB} Endow.},
    pages   = {2363--2377},
    year    = {2024}
}
1
2
3
4
5
6
7
@inproceedings{qiu2025tab,
    title   = {TAB: Unified Benchmarking of Time Series Anomaly Detection Methods},
    author  = {Xiangfei Qiu and Zhe Li and Wanghui Qiu and Shiyan Hu and Lekui Zhou and Xingjian Wu and Zhengyu Li and Chenjuan Guo and Aoying Zhou and Zhenli Sheng and Jilin Hu and Christian S. Jensen and Bin Yang},
    booktitle = {Proc. {VLDB} Endow.},
    year    = {2025}
}

1
2
3
4
5
6
7
@inproceedings{li2024TSMF-Bench,
    title     = {TSMF-Bench: Comprehensive and Unified Benchmarking of Foundation Models for Time Series Forecasting},
    author    = {Zhe Li and Xiangfei Qiu and Peng Chen and Yihang Wang and Hanyin Cheng and Yang Shu and Jilin Hu and Chenjuan Guo and Aoying Zhou and Christian S. Jensen and Bin Yang},
    booktitle = {SIGKDD},
    year      = {2025}
}

Contact

If you have any questions or suggestions, feel free to contact: